RC23694 (W0508-091) August 16, 2005 Computer Science IBM Research Report

This document re-evaluates the software stack in the light of para-virtualization technology and hypervisor support within next generation processors and operating systems. We describe an infrastructure enabling the use of logical partitions (LPARs) for the execution of stand-alone applications along side traditional operating systems. The design goal is to provide an environment allowing normal users to execute, interact and manage these custom kernels in much the same way they would with typical applications. The development environment is a set of modular component libraries providing necessary system services, and a familiar debug environment provided by exposing partition memory and control interfaces to a "controller" partition. We describe the implementation of our prototype using the IBM research hypervisor along with the Linux kernel and explore potential applications which could benefit from this new environment. 1

Glen or Glenda -- Empowering Users and Applications with Private Namespaces

Private name spaces were first introduced into LINUX during the 2.5 kernel series. Their use has been limited due to name space manipulation being considered a privileged operation. Giving users and applications the ability to create private name spaces as well as the ability to mount and bind resources is the key to unlocking the full potential of this technology. There are serious performance, security and stability issues involved with user-controlled dynamic private name spaces in LINUX. This paper proposes mechanisms and policies for maintaining system integrity while unlocking the power of dynamic name spaces for normal users. It discusses relevant potential applications of this technology including its use with FILESYSTEM IN USERSPACE[24], V9FS[8] (the LINUX port of the PLAN 9 resource sharing protocol) and PLAN 9 FROM USER SPACE[4] (the PLAN 9 application suite including user space synthetic file servers ported to UNIX variants)

Multi-Personality Network Interfaces

This report investigates the use of network interface (NIC) programming to provide multiple personalities or styles for the network interface, including ones that are similar to traditional storage and serial device interfaces. Doing so simplifies the implementation of other types of I/O access over the network at the link layer and helps advance the notion that the network can become the "I/O bus" of clustered systems such as blade servers. This type of support is unnecessary for traditional network environments in which all network access is done through the TCP/IP protocol stack. However, when using link-layer protocols and working directly with commodity network hardware, the mismatch between some types of I/O traffic and the behavior of the network interface becomes apparent, and it is then that the notion of alternative interface personalities becomes valuable. Although there are emerging interconnection architectures such as InfiniBand that offer solutions to this problem, they are currently beyond the range of commodity hardware. In addition, a previous, related effort on linklayer access to remote storage devices, the Ethernet Block Device (EBD), motivates this research. In the course of prototyping EBD, some of the obvious, but often overlooked, differences between network I/O and storage I/O became very apparent, and this effort attempts to solve these problems by concurrently offering multiple interface styles for the network adapter, including one that is better suited for block I/O operations

KNITS: Switch-based Connection Hand-off

This paper describes a mechanism allowing nodes to hand-off active connections by utilizing connection splicing at an edge-switch serving as a gateway to a server cluster. The mechanism is primarily intended to be used as part of a content aware request distribution strategy. Our approach uses an extended form of network address translation which maps inbound connection information (ie. address, port, and sequence number) to a separate outbound connection. A key difference in our approach is that while the switch performs network address translation and TCP splicing, the actual hand-off is triggered by the back-end nodes. This relieves the switch of performing any application layer responsibilities. Nodes may hand-off connections by first initiating a new connection to the destination and then sending a message to the gateway which splices the two connections together. The gateway modifies subsequent packet headers in order to create a transparent hand-off. This mechanism requires no modification to the operating system on the servers or the clients and supports HTTP/1.1 persistent connections and pipelined requests. To test our design, we implemented a soft-switch using Linux Netfilter which includes the extended network address translation. We provide some preliminary performance analysis and make recommendations for future work

Grave Robbers from Outer Space Using 9P2000 under Linux

This paper describes the implementation and use of the Plan 9 distributed resource protocol 9P under the Linux 2.6 operating system. The use of the 9P protocol along with the recent addition of private name spaces to the 2.6 kernel creates a foundation for seamless distributed computing using Linux. We review the design principles and benefits of Plan 9 distributed systems, go over the basics of the 9P protocol, describe 9P extensions to better support UNIX file systems, and show some example Linux distributed applications using 9P to provide system and application services. We conclude by providing a performance analysis of the protocol versus NFS for sharing a static file system

RC22602 (W0210-130) October 23, 2002 Computer Science IBM Research Report

This paper revisits the historical concept of link-layer, block-oriented storage networking in light of commoditybased, high-bandwidth local area networks. It examines how advances in data communication technology such as virtual LANs and switched gigabit Ethernet make it relatively simple and inexpensive to re-centralize the storage resources used by clusters of workstations and servers. In particular, the work is motivated by an interest in how to support scalable and efficient access to read-only and private read-write data such as root file systems, swap partitions, log files and static web pages. These techniques complement, but do not replace, higher level distributed file systems whose primary goal is to provide coherent access to shared read/write data. This paper describes the design and implementation of a very simple, link-layer protocol, the Ethernet Block Device (EBD), for accessing remote block devices. It compares the EBD prototype to a locally attached disk and to similar, network-based techniques that use TCP/IP such as the Linux Network Block Device (NBD), as well as higher level distributed file systems such as NFS. Functionally, the implementation is compared with a local disk to determine what restrictions and visible differences exist. The performance evaluation is based on a series of standard disk and file access benchmarks run on commodity servers and standard networking equipment. The performance results show that for large sequential reads and random reads and writes EBD generally outperforms comparative network storage technologies by 15% to 30%, and performance is best when the data set fits into the server's memory. On a benchmark that is very metadata-intensive and does small sequential operations, EBD and NBD perform similarly. On certain..

Console over Ethernet

While console support is one of the most mundane portions of an operating system, console access is critical, especially during system debugging and for some low-level system-administration operations. As a result, densely packed servers and server appliances have either keyboard/video/mouse (KVM) cabling and KVM switches or serial ports, cabling, and concentrators. Further increases in the density of server appliances and blades require eliminating these items. We did so in the design of a prototype dense-server-blade system that includes none of the standard external ports for console devices. Since the standard means for console accesss, KVM or serial, were not possible, we developed lowlevel software to redirect console activity to the Ethernet interface. This paper describes the console support over the network interface that we developed for both our LinuxBIOS-based boot-time firmware and the Linux operating system. We refer to this code as console over Ethernet or etherconsole. We found it invaluable in debugging and evaluating our prototype server blades. We also describe ways of extending our work to make it more transparent to existing firmware and operating systems